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ABSTRACT 



This document reports on the first year of a three year 
course of study for high school students in New York state known as the 
Science Research Program (SRP) . SRP affords students the opportunity to 
conduct scientific research in conjunction with research scientists and 
professionals in the field. The program implementation incorporates the 
student as the primary initiator of research, the teacher as facilitator of 
the scientific method, and a scientist to serve as the student's research 
mentor. This evaluation explores four areas: (1) the effectiveness of teacher 

and student recruitment and retention efforts; (2) the effectiveness of the 
teacher training component of the program; (3) the process of program 
implementation during year one; and (4) the effectiveness of the program as 
defined by year one outcomes. Appendices include the teacher interview 
protocol, symposia observation form, and the assessment rubies. (DDR) 
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SCIENCE RESEARCH PROGRAM 
EXECUTIVE SUMMARY 
Year One 

Evaluation Consortium 

University at Albany, State University of New York 
Albany, New York 12222 

The Science Research Program is a unique three year course of study offered to high school 
students across New York State. The program affords students the opportunity to conduct 
scientific research in conjunction with research scientists and professionals in the field. The 
program implementation incorporates the student as the prime initiator of research, the teacher as 
the facilitator of the scientific method, and a scientist to serve as the student’s research mentor. 
Students utilize communication technologies to facilitate their research. 

As part of the first year implementation, the Evaluation Consortium at the University at Albany 
was asked to assist in conducting a formative evaluation of the Science Research Program. The 
purpose of the evaluation was to document stakeholders’ perceptions of four areas: (1) the 
effectiveness of teacher and student recruitment and retention efforts; (2) the effectiveness of the 
teacher training component of the program; (3) the process of program implementation during 
year one; and (4) the effectiveness of the program as defined by year one outcomes. The 
evaluation team conducted interviews and surveys with students, teachers, and administrators. 
The following are the major findings of the first year of the program. 

Effectiveness of Teacher and Student Recruitment and Retention Efforts 



Teachers, administrators, and students offered their perceptions of the goals and purposes of the 

Science Research Program including: 

• having students do the process of science; “to find out that science is a verb not just 
something that you know; you do science not just learn science,” 

• having students gain in depth knowledge in a particular scientific area through independent 
and applied learning, and 

• improving student work habits such as time management skills. 

Perceptions of the recruitment and selection processes were as follows: 

• the recruitment process generally included teacher presentations to science classes and was 
perceived as functioning adequately, 

• the selection process ranged from simply enrolling students to formal application and 
interviews of students, 

• generally students selected for the program had a range of ability levels, 

• the most important student characteristics for selection were motivation and self-discipline, 
and 

Across the program the average attrition was minimal; some reasons for attrition were that the 

program was more demanding than the student had anticipated and that the student did not come 

up with a testable hypothesis. 
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Effectiveness of the Training Component 

The majority of the teachers reported that the training was outstanding. Their perceptions of the 
most useful aspects of the three week summer training were: 

• the hands-on approach to learn how to evaluate and facilitate the course components, and 

• the training manual and portfolio method of evaluation. 

Teachers listed areas in which more information would be helpful including: 

• using DIALOG, 

• assisting students with time management, 

• aiding in the student-mentor relationship, 

• utilizing basic statistics, 

• assisting students in reading journal articles, and 

• organizing the symposium. 

A majority of teachers found that the support groups were helpful. Teachers perceived the most 
useful aspects of the ongoing support groups were: 

• gaining feedback on how to solve specific problems, and 

• networking for mentors and journals. 

Suggested changes for the support groups included: 

• shifting to topic specific meetings in which participants could learn specific techniques, and 

• increasing time spent interacting with other teachers, e.g. a professional day. 

Program Implementation 

Students reported on the following positive aspects of their participation in the SRP: 

• experiencing a sense of accomplishment, 

• finding helpful resources (e.g. journal articles, experts, and mentors), and 

• gaining recognition for efforts. 

Students perceived that the following tasks were easy to master: 

• finding articles with DIALOG, 

• giving oral presentations after constructive feedback, and 

• writing a literature review. 

Assistance which the students indicated made tasks easier was: 

• being surrounded by others that did them earlier, and 

• having set guidelines. 

Students found the following tasks frustrating and/or difficult: 

• finding articles due to inaccessibility to a university library or difficulty with interlibrary loan, 

• contacting authors of journal articles, finding a mentor, and locating a research facility, 

• creating 5 timelines and completing tasks on time especially for the symposium. 
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• generating a testable hypothesis (47% reported a high level of support in this area), 

• giving the first oral presentation (56% reported a high level of support in this area), and 

• reading dense journal articles without sufficient background knowledge 

Students desired additional assistance in: 

• obtaining background information in order to better understand journal articles (33% reported 
a high level of support in this area), 

• finding a mentor (3 1 % reported a high level of support in this area), 

• managing deadlines (33% reported a high level of support in this area), and 

• sustaining motivation when the work gets exceptionally difficult. 

Teachers offered the following feedback on what facilitates program implementation: 

• having sufficient preparation time, 

• maintaining support from administrators, parents, and program administrators, 

• maintaining the district's financial support, and 

• having other teachers and the librarian available to assist students. 

Teachers indicated that the following were barriers to program implementation: 

• having inadequate library resources, 

• lack of necessary communication systems in the classroom (e.g. DIALOG, telephone), 

• insufficient time scheduled for program (some teachers lack a modified workload), and 

• lack of support from others in the science department. 

Teachers commented on the degree of support for the SRP received from various sources: 

• 100% reported sufficient to a high degree of support from administrators, 

• 56% reported adequate support from other science teachers, 

• 1 00% reported a high degree of support from the parents of their students, and 

• half of the teachers interviewed reported that the community was unaware of SRP. 

Year One Outcomes 



A number of positive outcomes support the effectiveness of the SRP in its first year of 
implementation in the various school districts. 

The frequency of student progress assessment ranged from bimonthly to four times per semester. 
Teachers reported utilizing the following methods of assessment: 

• student-teacher conference using rating sheets provided in workshop, 

• peer evaluation of oral presentations using rating sheets provided in workshop, 

• day to day supervision of student time management, and 

• portfolio review. 
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Teachers, administrators, and students identified the following skills to be improved as a result of 
participation in the SRP: 

• reading for meaning, 

• writing a report and/or abstract, 

• organizing and planning, 

• communicating effectively orally, 

• using technology effectively, and 

• interpreting data. 

Students were reported to improve in the following: 

• knowledge of conducting scientific research, and 

• knowledge of science. 

Reviews of student portfolios yielded the following observations: 

• the completion of the lab notebook, goals sheets, abstracts, and the e-mail and DIALOG logs 
were highly variable among the schools, 

• 87% of the portfolios included articles from primary sources as required by the SRP, 

• telephone and meeting logs were minimally completed, 

• student presentations were a major component of the portfolio including notes, graphics, and 
completed peer review forms, 

• no students had entries in the grants section, and 

• few students included the rules for competitions as part of their portfolios. 

Observations of symposia yielded the following: 

• every student presented a poster with word processed information and graphic analyses of the 
six sections of the scientific method, 

• students gave oral presentations of a literature review and/or experimental findings lasting 
fifteen minutes; in some schools, every student gave an oral presentation, 

• over half of the students synthesized several scientific journal articles in their literature 
review, and 

• each symposium included welcoming remarks and a keynote address. 

Administrators perceived that the SRP fit in with New York State Education Standards in the 
following ways: 

• meeting the Math, Science, and Technology initiative, 

• obtaining Regents credit as an additional science elective rather than a replacement science 
course, and 

• meeting the research component of the standards. 

One hundred percent of the administrators and teachers advocated for the long term viability of 
the SRP. All noted that the program assisted students in enhancing cognitive skills and work 
habits as well as providing a broader range of curriculum options to students. 
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Introduction 



The year one evaluation of the Science Research Program (SRP) focused on issues related to 
teacher training, implementation of year one academic activities, and documentation of year one 
outcomes. Specific evaluation objectives and questions are as follows: 

Objective One: To assess the effectiveness of teacher and student recruitment and 
retention efforts. Specifically to address the following: 

1 . What are the initial perceptions of the educational goals of the SRP? 

2. For whom is the program considered appropriate? 

3. How are students recruited and why do they join? 

4. How does student attrition impact the SRP and what retention efforts are made? and 

5. How are teachers recruited? 

Objective Two: To assess the effectiveness of the teacher training component of the 
program. Specifically to address the following: 

1 . How effective is the summer training workshop? and 

2. How effective are the teacher support groups? 

Objective Three: To assess the process of program implementation during year one. 
Specifically to address the following: 

1 . How did students perceive the initial year of the SRP including joys and frustrations 
and relationships with teachers and mentors? and 

2. How did teachers perceive the process of implementing SRP during year one, 
including supports and barriers to implementation, and student needs? 

Objective Four: To assess the effectiveness of the program as evidenced by year one 
outcomes. Specifically to address the following areas: 

1 . What methods are teachers using for student assessment? 

2. What are the perceived outcomes of students participation in the SRP and what skills 
are being mastered? 

3. Does a review of student portfolios support perceptions of student outcomes? 

4. Does a review of student symposia support perceptions of student outcomes? 

5 . What are the perceived changes to the educational system as a result of implementing 
the SRP? 
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Methodology 



Eight schools in their first or second year of Science Research Program implementation were 
visited by the evaluation team. The eight schools represented a cross-section of urban, suburban, 
and rural districts. District enrollment ranged from 481 to 5716 students. Enrollment in the 
Science Research Program ranged from 4 to 34 students. 

While on site, individual interviews were conducted with the Science Research teachers, 
administrators, and students. The eight administrators interviewed consisted of central 
administrators, science department heads, and building administrators. Sophomore, junior and 
senior students were interviewed in groups. A random sample of student portfolios were also 
examined at each site. 

In addition to site visits, surveys were mailed to 17 schools offering the Science Research 
Program. Students, teachers, and administrators from 1 1 of these schools responded. Eight of 
these 1 1 schools were also site visit schools. Three symposia were attended by evaluators. 

Following are the results of the Year One evaluation. The information collected is summarized 
under specific objective questions within objectives: 



Objective One: 

Effectiveness of teacher and student recruitment and retention efforts 



Perceptions of the Goals and Purpose of the SRP 

Teachers, students, and administrators were asked during interviews what they perceived the 
goals and/or purposes of the program to be. The perceptions of the three stakeholder groups 
were coordinated with the goals outlined by the project directors. 

Academic 

• to have students perform the process of science; "to find out that science is a verb, not just 
something that you know; you do science, not just learn science," 

• to learn how to become a scientist by framing a question, making observations, assimilating 
information, and drawing conclusions, 

• to gain depth of knowledge in a particular area through real life application of learning, 

• to improve reading, writing, and oral presentation skills, and 

• to prepare for college and obtain college credit. 

Work Habits 

• to learn time management skills, and 

• to be self-driven. 

Personal 

• to improve self-confidence, 

• to gain autonomy in learning, 

• to create a sense of accomplishment in students, 

• to gain an open mind, 

• to learn to accept an incorrect hypothesis, and 

• "to create an informed citizenry and develop the confidence to ask a question and find the 
answer to it.” 
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Teachers cited the following as key components of the SRP: 

• the use of the scientific method in a broader way than in a preestablished lab that is directed 
to one right answer, and 

• the gaining of depth over breadth in studying a scientific topic. 



Perceptions of the Population Served 

Teachers, students, and administrators were asked to describe the type of student best served by 
the SRP. They described the students as: 3 

• a broad range of students, usually the higher caliber students, 

• highly motivated and self-disciplined, 

• having a genuine interest in a scientific topic, and 

• willing to find out their own answers and take risks; e.g. " an attitude of wanting to do rather 
than what do I need to know to get an A" 

Recruitment and Selection of Students and Teachers 



Students, teachers, and administrators consistently reported general satisfaction regarding 
student recruitment and selection process. Table 1 summarizes their perceptions. 



Table 1. 

Selection and Recruitment Process 





Teacher 


Responses %: 




Administrator Res 


Donses %: 


Process 


Working 

Well 


Adequate 


Needs 

Extra 

Work 


Not Area 
of 

Emphasis 


Working 

Well 


Adequate 


Needs 

Extra 

Work 


Not Area 
of 

Emnhack 


Recruitment of teacher 
into the program 


56 


0 


ii 


33 


50 


30 


0 


■-4 III piltljlj 
20 


Identification of 
prospective 

students for the program 


11 


56 


33 


0 


50 


30 


20 


0 


Spring recruitment 
meeting 
Freshman year 


11 


33 


44 


11 


30 


30 


40 


0 


Student application 
process 


11 


33 


44 


11 


NA* 


NA 


NA 


NA 


Fall meeting with students 


44 


33 


0 


0 


30 


70 

50 


0 

10 


0 

0 


Process of retaining 
students in program 


44 


44 


11 


0 


40 


*NA - Not asked 


1 



The recruitment process was perceived as functioning adequately. Specific findings derived from 
qualitative and quantitative data are as follows: 

Rjicruinrtent 

• in three-quarters of the schools visited, the SRP teacher briefly presented the program to 
students in other science classes, 

• of those presentations, the overwhelming majority were made to all levels of students and 
some were made to honors students only, 
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Teachers cited the following as key components of the SRP: 

• the use of the scientific method in a broader way than in a preestablished lab that is directed 
to one right answer, and 

• the gaining of depth over breadth in studying a scientific topic. 

Perceptions of the Population Served 

Teachers, students, and administrators were asked to describe the type of student best served by 
the SRP. They described the students as: 

• a broad range of students, usually the higher caliber students, 

• highly motivated and self-disciplined, 

• having a genuine interest in a scientific topic, and 

• willing to find out their own answers and take risks; e.g. " an attitude of wanting to do rather 
than what do I need to know to get an A" 

Recruitment and Selection of Students and Teachers 

Students, teachers, and administrators consistently reported general satisfaction regarding the 
student recruitment and selection process. Table 1 summarizes their perceptions. 

Table 1. 

Selection and Recruitment Process 





Teacher 


Responses 


%: 




Administrator Res 


ponses %: 


Process 


Working 

Well 


Adequate 


Needs 

Exjtra 


Not Area 
of 


Working 

Well 


Adequate 


Needs 

Extra 


Not Area 
of 








Work 


Emphasis 






Work 


Emphasis 


Recruitment of teacher 
into the program 


56 


0 


ii 


. 33 


50 


30 


0 


20 


Identification of 
prospective 

students for the procrm 


11 


56 


33 


0 


50 


30 


20 


0 


Spring recruitment 
meeting 
Freshman year 


11 


33 


44 


11 


30 


30 


40 


0 


Student application 
process 


11 


33 


44 


11 


NA* 


NA 


NA 


NA 


Fall meeting with students 


44 


33 


0 


0 


30 


70 


0 


0 


Process of retaining 
students in program 


44 


44 


11 


0 


40 


50 


10 


0 



*NA = Not asked 



The recruitment process was perceived as functioning adequately. Specific findings derived from 
qualitative and quantitative data are as follows: 

R^crmtmenj 

* in three-quart . . s of the schools visited, the SRP teacher briefly presented the program to 
students in other science classes, 

• of those presentations, the overwhelming majority were made to all levels of students and 
some were mare to honors students only, 
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superior ^uman^Iati^^A^^isttators^d^te^^ers'^^hl^I^^^gg an< ^ or 

recruitment process. u e ni y satisfied with the teacher 

^Diect.ve I wo: Effectiven ess of the Teacher Training Comnn^ T 
Summer Training Workshop 

One year after the training, the teachers were asked to offer feedback nn th^ tr h e 

supper, efforts offered by the program. Table 2 provides a summa^ of .he da^ ” 8 ““ 

rj . , Table 2 

Tea chers* Perceptions of Support Provided bv Training and „ 



Support 


Outstanding 

% 


Sufficient 

% 


Insufficient/ 
No SuDnort % 


Orientation sessions 


22 

88 

13 


63 

13 

63 


13 

0 

25 


Summer training session 


Support group meetings 



of 



the quantitative and qualitative data "" ° UtStandln8 ' Examinat 

M osj useful aspects ofthe summe r training workshop s 

• ev“XTS?fa^r aSe ‘ he ‘ eaCher ’ S UndCTSandi "« of >he student experience. 

• emphasizing the scientific method as the center of the curriculum. 

A reas in which more info r mation wouldbe hel pful 

• using DIALOG, a — 

• organizing the symposium, 

• teaching students to read journals, 

• SRP in .heir junior or senior year, 

• conducting basic statistics. 

Ch anges in format or content suggested 

• increasing time for teachers to put together their presentations 

• ^. a P. tln S to students of differing reading and writing skills 
altering documentation of first year activities from lab book method. 

Teacher Support Groups 

Pr °fT T '“on and 

solutions. Sixty-tliee percent and 1 the shann « of Problems and 

13% found .he level of support ,o be outstanding. ^cff^X." taSSKSo^' 
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Mosj useful aspects oj_ the support groups 

• gaining feedback on how to solve specific problems, and 

• networking for mentors and journals. 

Areas in which more information would be helpful 

• organizing the symposium, and 

• sharing information arid ideas. 



Change y in formal QL content suggested 

• a shift to topic specific meetings was desired by the vast majority of interviewees in order to 

learn specific tools or techniques, ur r 0 

• less time spent reporting on progress, 

• more time to interact with other teachers e.g. a professional day, and 

• better facilitation of the meetings so that participants stay focused on the SRP model. 



Objective Three: Process of Program Implementation 



Seven of the eight site visit schools responded to a sixteen question checklist that 
main aspects of a successful program. These findings are summarized in Table 3. 



addresses the 



Table 3 

SRP Implementation Success Checklist 



Goal 


Yes 

% 


No 

% 


Not 

Applicable% 


Spring meeting with prospective 


86 


14 


0 


students 






V 


Midyear list of prospective students 


57 


43 


1 


Teacher availability in summer 


86 


14 


1 

n 


Ten articles read by start of school 


100 


0 


u 

n 


Five biweekly assessments per 


71 


29 


u 

0 


quarter 








Student-teacher conferences by 


100 


0 


n 


week three 






u 


Students complete three DIALOG 


71 


29 


0 


searches 






u 


Prospective mentor contact by April 


57 


43 


0 


Students know role in Symposium 


86 


14 


u 

0 


by April 








All student portfolio section kept up 


43 


57 


0 


Opportunities for out of class 


71 


29 


yj 

0 


presentations 








Students attended available 


100 


0 


0 


symposia 








Students produced quality poster by 


100 


0 


0 


year one 








Second and third year students get 


43 


14 


43 


college credit 








Second and Third year students 


71 


0 


29 


enter competitions 








Students participated in symposium 


86 1 


14 


0 



O 
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Specific findings include the following: 

the basic components of the SRP were generally met at all reporting sites 

• S/ ltm / nt P rocedures var Y from the recommended procedures 

• 1 00 /o of students read 1 0 articles prior to the beginning of the school year 

greater attention may peed to be directed toward the maintenance of the student portfolio 
?rS/ sc f hool , held an ann ual symposium in which each student produced a poster 

• n^Pn^ f t StUdentS | T k advan ! age of opportunities to attend available science symposia and 
presentations including attending a neighboring school's symposium 

• 57% of students attempted to procure a mentor by April in the first year of the course 

• the intended assessment process appears to be in place, and 

• 71 /o of second and third year students entered competitions. 

Student Perceptions 



Students were asked to comment on the joys and frustrations that they experienced conducts 
research as well as the aspects of the program that were easy or difficult to master. 8 



Joys 

• experiencing a sense of accomplishment, 

• finding helpful resources, and 

• gaining recognition for efforts. 



Tasks that_ were easy to master 

• finding articles with DIALOG, 

• oral presentations after constructive feedback, and 

• literature review. 



What made tasks easier 

• being surrounded by others that did them previously and 

• having set guidelines. 



Frustrations 

• finding articles via interlibrary loan and DIALOG, 

• contacting authors of journal articles and finding a mentor, 

• getting tasks done on time especially for the symposium, and 

• scheduling conflicts with other courses or activities. 



Tasks that were difficult 

• generating a testable hypothesis, 

• the first presentation of an article, 

• reading dense journal articles without sufficient 

• establishing a mentor relationship, and 

• locating a research facility. 



background knowledge, 



Students also commented on their relationships with their teachers and their mentors durine their 

S*" 8 - Ta , bles4 ®<1 5 present perceptions of support from teachers Ld 
mentors ror both sophomore and junior students. 
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Table 4 

St udents’ Reported Level of Support Receiv e d from Teachers and Mentor. 

During Their Sophomore Year 



Sophomore Year 


Teach 

High 


er Support 
Average 


t% 

Low 


Not 

Applicable 


Mentc 

High 


>r Support 
Average 


% 

Low 


Not 

Annlimhlp 


Mastering DIALOG 
computer searches 


46 


29 


3 


22 


1 


2 


2 


95 ~ 


Locating and retrieving 
20 articles using 
DIALOG 


33 


31 


11 


11 


0 


4 


1 


95 


Presenting one of the 20 
articles to class 


56 


26 


2 


16 


3 


4 


1 


92 


Developing statement of 
intended research 


47 


29 


7 


17 


15 


7 


4 


74 


Contacting author(s) of 
.retrieved articles to 


33 


22 


9 


36 


5 


2 


5 


88 


begin mentor 
relationship 


















Locating a research 
facility 


31 


15 


6 


48 


12 


6 


1 


81 


Creating five timelines 


31 


26 


11 


32 




A 




91 

86 


Creating posterboard 
detailing intended 
research and initial 
findings 


56 


22 


5 


17 


Z, 

6 


6 


2 




Beginning research 


54 


17 


4 . 


4 


18 


4 


2 


76 



BEST COPY AVAILABLE 
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Table 5 

St udents’ Reported Level of Suppor t Received from Teachers and Mentors 
During Their J unior Year (Accelerated Work) 



Junior Year 


Teach 

High 


er Support 
Average 


% 

Low 


Not 

Applicable 


Mento 

High 


>r Support 
Average 


% 

Low 


Not 

AnnliP/i hip 


Sustaining bi-monthly 
communication with 
mentor 


12 


6 


2 


80 


8 


4 


3 


85 


Continuing literature 
reviews at a rate of 
approximately 20/year 


11 


12 


3 


74 


6 


1 


4 


89 


Redefining hypothesis 
based on literature 


17 


5 


3 


75 


9 


5 


3 


83 


Presenting findings to 
teacher, the class, and 
school community 


20 


4 


4 


72 


5 


5 


2 


88 


Orientation to statistical 
software 


8 


3 


3 


86 


3 


2 


2 


93 


Constructing five time 
lines 


8 


7 


7 


78 


0 


1 


2 


97 



They noted the types of assistance received from the teacher. 



Academic^ 

• suggested research tools, methods, and focus 

• Sf V0Cabulary terms and P rovided scientific knowledge in translating a journal 

• assisted in finding a mentor (33% of students reported a high level of support in this area), 

• 56 /o reported getting a high level of support in oral presentations, and 

• 47% reported getting a high level of support in generating a testable hypothesis. 



Work Habits 

helped with time management (3 1 % of students reported a high level of teacher support). 
Personal 

• supportive and understanding of frustration. 



^caSj/c 1S ° COmmented on the additi °nal assistance that the teacher could provide: 

• finding a mentor, 

• helping with the writing of the paper, and 

• providing more background information on their topic. 

Work Habits^ 

• managing deadlines. 
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Personal 

• providing encouragement when the work gets difficult. 

Students noted the types of assistance received from the mentor 

Academic 

• designing the experiment, 

• explaining theory, background, terminology, and 

• providing constructive criticism on the paper and presentation. 

Personal Support 

• encouraging the student, and 

• attending the symposium. 



Students reported on the additional assistance that the mentor could provide 
Academic K 

• designing the experiment, 

• analyzing data statistically, and 

• gauging whether student has sufficient information for the project. 



' n §^ ne f ral - -entorswere not reported to be providing a high level of support. This finding mav 
confounded by the fact that most students do not have mentors in this point of their research! 



Teacher Perceptions 



The eight teachers offered feedback on what facilitates and what impedes program 
implementation. Listed below are factors which facilitate program implementation 

• having sufficient preparation time, 

• maintaining support of administrators, parents, and program administrators 

• maintaining district’s financial support, and 

• gaining the support of other teachers and librarians in assisting students. 



The following factors impede program implementation: 

Inadequate resources/ support 

• inadequate library resources: interlibrary loan is insufficient; not near university 

• lack of communication systems in the classroom (e.g. telephone DIALOG) 

• insufficient time scheduled for the program, and 

• inadequate support from colleagues in the science department 



library, 



The Science Research Program recommended that teachers be assigned four course workloads 
however, that was not the case for some of the schools visited. In addition, in classroom 
communication systems were not present in some schools as recommended by the proposal. 



Objective Four: Outcomes 



A number of positive outcomes were reported by students, teachers, and administrators which 
supported the effectiveness of the SRP in its first year of implementation. These included 
utilization of alternative means of assessment, student outcomes, and systems outcomes. 

Assessment 

inctude^ rep0rted im P lementat i°n of suggested modes of alternate assessment. Findings 
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Frequency 

• some assessed student progress 3 to 4 times per semester, and 

• a few of the teachers conducted bimonthly evaluations. 

AssessmerU Activities 

• student-teacher conference using rating sheets provided in the summer training workshop, 

• peer evaluation of oral presentations, 

• day to day time management supervision, and 

• review of entire portfolio. 

Student Outcomes 

The students, teachers, and administrators outlined many programmatic benefits to students. 
These findings are summarized in Tables 5 and 6. 

Table 6 

Agreement of Presence of Recognized Outcomes of the SRP: 

Percentage of Teachers, Administrators, and Students 
Perceiving the Outcome to be Present 





Teacher % 


Administrator % 


Student % 


The program has improved 
knowledge about conducting 
science research 


100 


100 


99 


The program has increased 
knowledge about science 


100 


100 


97 


The program is interesting to me 


100 


100 


98 


The program has helped to 
improve: 

creative thinking ability 


100 


100 


96 


problem solving skills / 


100 


100 


97 


communication skills 


100 


100 


98 


time management skills 


100 


90 


92 


Due to the program, motivation 
for learning has increased for: 
science 


88 


100 


95 


other school subjects 


51 


70 


82 


Self-esteem has increased 


100 


100 


86 


The class size is good 


100 


100 


94 


The program challenges 


100 


100 


99 


The Program has changed the 
concept of science education 


75 


80 


NA* 


I support continuation of the 
program 


100 


100 


NA 



*NA = Not Asked 
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Table 7 

Teacher, Administrator and Student Perceptions of Skills Acquired 

by Students in the SRP 



Skill 


Student % 


Teacher % 


Admin% 


Reading for meaning 


85 


100 


91 


Problem solving 


68 


75 


82 


Reasoning skills 


70 


88 


73 


Writing a report/abstract 


89 


100 


82 


Organization and planning skills 


85 


88 


73 


Thinking creatively 


63 


63 


27 


Communicating effectively orally 


89 


100 


82 


Communicating effectively in writing 


68 


100 


82 


Identifying the difference between a fact 
and an opinion 


46 


88 


73 


Making an inference 


63 


75 


64 


Anticipating possible problems and 
finding the solution , s 


71 


50 


45 


Effective use of technology 


81 


88 


91 


Interpretation of data 


81 


88 


73 


Outlining or notetaking 


81 


50 


82 


Summarizing or paraphrasing 


72 


63 


82 


Collecting data 


56 


75 


55 


Reading graphs and charts 


71 


100 


91 


Time management 


71 


88 


64 


Real life application of science 


72 


88 


91 



At least 80% of each of the three stakeholder groups believed that the following skills were 
enhanced by the SRP: 

Academic 

• reading for meaning, 

• writing a report and/or‘abstract, 

• effective oral communication, and 

• effective use of technology. 



In addition, the vast majority of the three stakeholder groups strongly agreed that the students’: 

• knowledge of conducting scientific research improved, and 

• knowledge of science improved. 

Comments generated during the interviews offered additional academic student outcomes such 
as: 

• meaningful, in-depth learning of an individualized topic that motivates the student, 

• knowledge of how to digest and relate information from literature, 

• accrual of critical thinking skills, and 



O 

ERLC 
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Table 7 

Teacher. Administrator and Student Perceptions of Skills Acquired 

by Students in the SRP 



Skill 


Student % 


Teacher % 


Admin% 


Reading for meaning 


85 


100 


91 


Problem solving 


68 


75 


82 


Reasoning skills 


70 


88 


73 


Writing a report/abstract 


89 


100 


82 


Organization and planning skills 


85 


88 


73 


Thinking creatively 


63 


63 


27 


Communicating effectively orally 


89 


100 


82 


Communicating effectively in writing 


68 


100 


82 


Identifying the difference between a fact 
and an opinion 


46 


88 


73 


Making an inference 


63 


75 


64 


Anticipating possible problems and 
finding the solution 


71 


50 


45 


Effective use of technology 


81 


88 


91 


Interpretation of data 


81 


88 


73 


Outlining or notetaking 


81 


50 


82 


Summarizing or paraphrasing 


72 


63 


82 


Collecting data 


56 


75 


55 


Reading graphs and charts 


71 


100 


91 


Time management 


71 


88 


64 


Real life application of science. ; 


72 


88 


91 



At least 80% of each of the three stakeholder groups believed that the following skills were 
enhanced by the SRP: 

Academic 

• reading for meaning, 

• writing a report and/or abstract, 

• effective oral communication, and 

• effective use of technology. 



In addition, the vast majority of the three stakeholder groups strongly agreed that the students’: 

• knowledge of conducting scientific research improved, and 

• knowledge of science improved. 

Comments generated during the interviews offered additional academic student outcomes such 
as: 

• meaningful, in-depth learning of an individualized topic that motivates the student, 

• knowledge of how to digest and relate information from literature, 

• accrual of critical thinking skills, and 
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• students had no entries in the grants and essays sections, 

• few had rules for competitions as part of their portfolios, and 

• most students did not have final papers because they are not at that stage of their research 
being mostly sophomores and juniors. 



Table 8 

Percent of Portfolios Reviewed with Content Present 



Contents 


None 

* 


Minimal 


Normative 


Superior 


Not 

Applicable 


Lab notebook 


38 


13 


6 


44 


0 


Goal sheets 


19 


6 


31 


44 


0 


e-mail log, DIALOG log 


13 


19 


31 


38 


0 


Journal articles 


6 


0 


50 


44 


0 


Telephone log 


50 


44 


0 


0 


6 


Meetings log 


56 


44 


0 


0 


0 


Assessment 


25 


0 


44 


31 


0 


Grants 


100 


0 


0 


0 


0 


Presentations 


6 


13 


25 


56 


0 


Honors and Awards 


44 


0 


25 


31 


0 


Abstracts 


44 


0 


19 


38 


0 


Final paper 


0 


0 


0 


13 


88 


Rules for competitions 


69 


0 


32 


0 


0 


Log of time commitments 


32 


0 


4 


32 


0 


Essay on excellence 


100 


0 


0 


0 


0 


Essay on research direction 


100 


0 


0 


0 


0 



Symposium Observations 

Three symposia conducted by students in their first year of the program were observed by 
evaluators. The students presented an extremely diverse range of topics in the natural and social 
sciences. 

Presentation Format 

• the symposia were attended by 25 to 60 people, 

• each symposium included welcoming remarks and a keynote address, 

• students received awards and certificates, 

• students gave oral presentations lasting fifteen minutes, 

• all students used graphic representations in their presentations, 

• the total group of presentations clearly conveyed the purpose, materials, methods, results, 
discussions, and conclusions of the literature reviewed and/or the experiments performed by 
the students, 

• the quantity and quality of journals synthesized in the literature reviews were variable, 

• over half of the students synthesized several scientific journal articles in their literature 
review, the remaining reviewed only one article and/or utilized anecdotal or nonprimary 
sources, and 

• two symposia included question and answer sessions with the audience. 

Poster Formal 

• every student presented a poster, 
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• wordprocessed information and graphic analyses were attached to the showboards, and 

• the total group of presentations clearly conveyed the purpose, materials, methods, results, 
discussions, and conclusions of the literature reviewed and/or the experiments performed by 
the students. 

Creativity was displayed in the symposia. Programs printed on heat sensitive paper were handed 
to audience members in one symposium. Students handed out business cards with their name, 
telephone number, e-mail address, and topic in another symposium. Please refer to photographs 
in the Appendix. 

System Outcomes 

Teachers commented on the degree of support received from various sources in implementing 
the program that is highlighted below. Table 9 presents a summary of teacher responses. 

Table 9 

Teacher and Administrator Perceptions of Changes Made 
and Supports Received from Various Organizations 





Teacher Responses % 


Administrator Responses % 


Support 


Outstanding 


Sufficient 


Insufficient/ 

NoSupport 


Outstanding 


Sufficient 


Insufficient/ 
No Support 


Adjustment of Teacher 
Load 


33 


22 


44 


45 


45 


5 


Establishment of 
communication networks 


0 


44 


56 


45 


55 


0 


Teacher computer for 
DIALOG, e-mail and 
Internet 


22 


33 


44 


55 


45 


0 


Long distance phone line 


22 


33 


44 


55 


45 


0 


Course list in course 
catelog 


, 56 


44 


0 


88 


13 


0 


Support from other 
teachers 


22 


56 


22 


45 


36 


18 


Support from school 
counselors 


0 


100 


0 


45 


45 


9 


Support from 
administrators 


78 


22 


0 


72 


27 


0 


Support from mentors in 
the community 


25 


38 


38 


45 


36 


18 


Support from parents 


50 


50 


0 


56 


44 


0 


Support from school board 


NA* 


NA 


NA 


73 


27 


0 


Support from State 
Education 


NA 


NA 


NA 


25 


50 


25 



*NA = Not Asked 
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• 99% reported support from administrators, 

• 85% reported support from other science teachers, 

• 1 00% reported a high degree of support from parents of their students, 

• Half of teachers interviewed reported that the community was unaware of the SRP, 

• 1 00% of administrators reported support from school boards, and 

• 88% of administrators reported that the course was listed in the course catalog. 

Administrators remarked on the advantages and drawbacks to having a SRP in their school. 

Advantages 

• the students learn self-discipline and responsibility, 

• students may eventually engage in research as a career, 

• the teacher is not burdened by needing an expertise in a particular subject area, 

• the program assists in improving students reading and communication skills, and 

• the skills in the program translate to all academic areas. 

Disadvantage^ 

• SRP does not reach enough students, yet it is too high level work to involve many students, 

• scheduling is difficult, particularly in smaller schools; it conflicts with many electives, and 

• SRP may not foster individual experimentation quickly enough. 

Administrators and teachers indicated their perceptions of changes and supports put into place to 
implement the program. 

Administratrators ’ perceptions of supports 

• most schools reported that the course is in the program of studies; some schools conduct the 
program after school so it is not currently in the program of studies, 

• most purchased minimal supplies and made budget changes, 

• many purchased lab equipment, journals, laptops, or software, 

• some added a telephone line, 

• some added DIALOG/Intemet account, 

• in a few schools, other teachers supplemented science research with their own expertise, e.g. 
a mini-course in statistics, and 

• buses provided for field trips to the library. 

Teachers’ and administrators' perceptions of the supports provided stood in sharp contrast. 
Teachers ’ perceptions of supports 

• reported insufficient to no adjustment of the teacher load, and 

• report insufficient establishment of communication networks (e.g. having a teacher computer 
for DIALOG, long distance telephone line, etc.). 

1 

Administrators reflected on the program's impact on the overall science curriculum: 

• stated that it was an expansion or addition in research exposure, 

• viewed students as curriculum experts in other science curriculums and saw it as an 
opportunity to transfer knowledge to other courses, and 

• reported that students continue to participate in the Regents curriculum which is driven by 
New York State Education Department. 

Administrators differentiated this course from other science courses because: 

• a body of knowledge is acquired through research rather than memorization of facts, 

• students work independently and are responsible for individual progress, 

• it is a more hands-on approach and a real life application of science, 
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• the teacher is a facilitator; "the director of education not the giver of information," 

• technology is integrated in a sustained fashion, and 

• students learn to communicate effectively orally. 

Respondents believed the program helps to fulfill New York State Education Standards in the 
following ways: 

• it meets the Math, Science, Technology initiative, 

• students can obtain Regents credit but it is an additional science course not a replacement 
science course, and 

• research is a component of the standards. 

When asked how it meets the needs of the student body as a whole, some administrators believed 
that the Science Research Program was geared for a specific sector of students and by its nature 
can not be a large progrpi. Another commented that it has the potential to foster greater respect 
and interest for science in the total student body. In one school system, the symposium was 
reproduced for the faculty so that they could become more aware of the use of technology. 

The Science Research Program expands the high school science curriculum by adding a unique 
elective which addresses the often neglected research component. In addition, it is reported by 
participants reported that the program not only improves students’ cognitive skills and 
knowledge of science, but also their work habits and personal attributes. Its continued 
implementation stands to further develop the scientific community. 
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Appendix A 

Student, Teacher and Administrator Surveys 



School District: 



Student Survey of Science Research Program 



As part of an effort to evaluate the Science Research Program, we are collecting perceptions 
about the effectiveness of the program. As a student involved in this program, your input is very 
important to us. Please read each of the following questions and provide us with your response. 
To maintain the confidentiality of your response, please place the survey in the accompanying 

envelope, seal it and return it to . 

***************************************************************************** 



About You 

1. Gender: Male Female 

2. Grade: 

3. Age: 

4. Number of years in the Science Research Program: 

5. Number of high school science courses you have already taken or are presently 

taking: 

6. Number of high school math courses you have already taken or are presently 

taking: 

7. Are you currently taking another science course this year besides the Science 

Research Program? If Yes, what is name of the course? 

Enrolling in the Science Research Program 

1. How did you find out about the Science Research Program? 



2. With whom did you talk when you were making the decision to enter the Science Research 
program? Please check all that apply. 

1. Parents/Caregivers 4. Administrators 

2. Teachers 5. School Counselors 

3. Peers 6. Other: 

3. Why did YOU decide to enter the program? 



ERIC 



3G 



About the Science Research Program 

The following statements are about the Science Research Program. Please indicate your level of 
agreement with each statement by circling the number that best represents your opinion. 

l=strongly agree; 2=agree; 3=slightly agree; 4=slightly disagree; 5=disagree; 6=strongly 

disagree 



1 . The program has improved my knowledge 
about conducting science research 

2. The program has increased my knowledge 
about science. 

3. The program is interesting to me. 

4. The program has helped to improve: 

a. my creative thinking ability. 

b. problem solving skills 

c. my communication skills. 

d. my time management skills. 

5. Due to the program, my motivation 
for learning has increased for: 

a. science 

b. other school subjects 

6. My self-esteem has increased due to the 
program. 

7. The class size is good. 

8. The program challenges me. 



Agreement Scale 



SA 

1 2 

1 2 

1 2 

1 2 

1 2 

1 2 

1 2 



1 2 
1 2 
1 2 

1 2 
1 2 



3 4 

3 4 

3 4 

3 4 

3 4 

3 4 

3 4 



3 4 
3 4 
3 4 

3 4 
3 4 



SD 

5 6 

5 6 

5 6 

5 6 

5 6 

5 6 

5 6 



5 6 
5 6 
5 6 

5 6 
5 6 



Please check any of the following that you have learned or improved through participation in the 
Science Research Program. 



_reading for meaning _ 

_problem solving 

_reasoning skills _ 

_writing a report/abstract __ 

_organization and planning skills _ 

_thinking creatively _ 

_communicating effectively orally _ 

_communicating effectively in writing _ 

-identifying the difference between a fact and an opinion 
_making an inference - using an observation to make a prediction 
—anticipating possible problems and finding the solution 



effective use of technology 
interpretation of data 
real life application of science 
outlining or notetaking 
summarizing or paraphrasing 
collecting data 
reading graphs and charts 
time management 
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About vour teacher and mentor 

Please indicate the level of support you received from your teacher and mentor in each of the 
following areas. Circle the number that best represents your opinion using the following scale. 
l=high level of support; 2=average level of support; 3=low level of support; 4=not 

applicable 



Sophomore year 

1. mastering DIALOG computer searches 

2. locating and retrieving 20 articles using DIALOG 

3. presenting one of the 20 articles to class 

4. developing statement of intended research 

5. contacting author(s) of retrieved articles to 
begin mentor relationship 

6. locating a research facility 

7. creating five timelines 

8. creating posterboard detailing intended 
research and initial findings 

9. beginning research 



Teacher 


Mentor 


1 2 3 


4 


12 3 4 


1 2 3 


4 


12 3 4 


1 2 3 


4 


12 3 4 


1 2 3 


4 


12 3 4 


1 2 3 


4 


12 3 4 


1 2 3 


4 


12 3 4 


1 2 3 


4 


12 3 4 


1 2 3 


4 


12 3 4 


1 2 3 


4 


12 3 4 



If you are in your second year of the Science Research Program (Junior year), please continue 
with items 10-15. If not, please go to item 16. 



Junior year 

10. sustaining bi-monthy communication with mentor 

11. continuing literature reviews at a rate of 
approximately 20 per year 

12. redefining hypothesis based on literature review 

13. presenting findings to teacher, the class, 
and school community. 

14. orientation to statistical software 

15. constructing five time lines 



Teacher 
12 3 4 

12 3 4 
12 3 4 

12 3 4 
12 3 4 
12 3 4 



Mentor 
12 3 4 

12 3 4 
12 3 4 

12 3 4 
12 3 4 
12 3 4 



16. In which of the above areas would you have liked more assistance (indicate area number)? 



About vour future plans 

1. What would you like to be doing 6 months after high school? 



2. What would you like to be doing 5 years after high school? 



3. Do you plan to go to college? If so, what do you plan to study? 



Other comments about the program? 



Thank you. 



School District: 



Teacher Survey of Science Research Program 



As part of an effort to evaluate the Science Research Program, we are collecting perceptions 
about the effectiveness of the program. As a teacher involved in this program, your input is very 
important to us. Please read each of the following questions and provide us with your response. 
To maintain the confidentiality of your response, please place the survey in the accompanying 
envelope, seal it and return it to . 

Gender: Male Female 

Ethnicity: 

Number of Years in Teaching: 

Number of Years Teaching in the Science Research Program: 



Support for the Science Research Program 

The following are areas in which you may have received support for the implementation of the 
Science Research Program. For each, please circle the number indicating the level of support you 
have received. 



Outstanding Sufficient 



1. Adjustment of teacher load. 1 2 

2. Establishment of communication 

networks. 1 2 

3. Teacher computer for DIALOG, 1 2 

email, and internet. 

4. Long distance phone line. 1 2 

5. Course list in course catalog. 1 2 



Insufficient 

3 

3 

3 

3 

3 



No Support 
4 

4 

4 

4 

4 



6. Support from other teachers. 1 

7. Support from school counselors. - 1 

8. Support from administrators. 1 

9. Support from mentors in the 

community. 1 

10. Support from parents. 1 



2 3 4 
2 3 4 
2 3.4 

2 3 4 
2 3 4 



Training 

Three basic types of training support were to be available to you under this program. For each 
type listed below, please circle the number indicating the degree of assistance it gave you for 
teaching the Science Research Program. 

Outstanding Sufficient Insufficient No Support 



1. Orientation sessions 

2. Summer training session 

3. Support group meetings 



1 2 3 
1 2 3 
1 2 3 



4 

4 

4 



Are there areas in which you would have liked additional training and support? If so, in what 
areas and in what form, would you have liked that support? 



Selection/Retention Process 

For each facet of the selection/retention 

how well the process is functioning. 

1 . Recruitment of teacher into the 
program 

2. Identification of prospective 
students for the program 

3. Spring recruitment meeting 
Freshman year 

4. Student application process 

5. Fall meeting with students 

6. Process of retaining students 
in program. 



the number indicating your impression of 



Is 


Needs Extra 


Not Area of 


Adequate 


Work 


Emphasis 


2 


3 


4 


2 


3 


4 


2 


3 


4 


2 


3 


4 


2 


3 


4 


2 


3 


4 



process, circle 



Working 

Well 



1 

1 

1 

1 

1 



When recruiting students, what would you say are the three most important characteristics for 
them to possess if they are to enroll in the program? 



How important is the student’s previous academic experience in science in the selection process? 



Communications 

Please rate your perception of the communication amongst the various program participants. 



Outstanding Sufficient 



1. Student & Teacher I 2 

2. Student & Mentor 1 2 

3. Teacher & Mentor 1 2 

4. Teacher & Administrator 1 2 

5. Teacher & School Counseor 1 2 

4. School & Parent 1 2 



Insufficient 

3 

3 

3 

3 

3 

3 



Not Happening 
4 
4 
4 
4 
4 
4 



Are there any specific areas in which you would like to see communication improve? 



About the Science Research Program 

The following statements are about the Science Research Program. Please indicate your level of 
agreement with each statement by circling the number that best represents your opinion. 
l=strongly agree; 2=agree; 3=slightly agree; 4=slightly disagree; 5=disagree; 6=strongly disagree 

Agreement Scale 



1. The program has improved students’ 
knowledge about conducting 
science research 

2. The program has increased students’ 
knowledge about science. 

3. The program is interesting to teach. 

4. The program has helped students’ 
to improve. 

a. creative thinking ability. 

b. problem solving skills 

c. communication skills. 

d. time management skills. 

5. Due to the program, student motivation 
for learning has increased for: 

a. science 

b. other school subjects 

6. Students’ self-esteem has increased 
due to the program. 

7. The class size is appropriate. 

8. The program challenges students. 

9. The program has changed the 
concept of science education 

10. 1 support continuation of the 
program 



SA 

1 2 3 



1 2 3 

1 2 3 



1 2 3 
1 2 3 
1 2 3 
1 2 3 



1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 



SD 

4 5 6 



4 5 6 

4 5 6 



4 5 6 
4 5 6 
4 5 6 
4 5 6 



4 5 6 

4 5 6 

4 5 6 

4 5 6 

4 5 6 

4 5 6 

4 5 6 



Please check any of the following that you believe students have learned or improved as a direct 
result of participation in the Science Research Program. 



reading for meaning _ 

_problem solving 
reasoning skills 

writing a report/abstract _ 

organization and planning skill _ 

thinking creatively 

communicating effectively orally _ 

communicating effectively in writing __ 

identifying the difference between a fact and an opinion 
making an inference - using an observation to make a prediction 
anticipating possible problems and finding the solution 



effective use of technology 
interpretation of data 
real life application of science 
outlining or notetaking 
summarizing or paraphrasing 
collecting data 
reading graphs and charts 
time management 



Of the above check-list, which do you think are most unique to this program? 

Thank you. 



School District: 

Administrator Survey of Science Research Program 

As part of an effort to evaluate the Science Research Program, we are collecting perceptions 
about the effectiveness of the program. As an administrator involved in this program, your input 
is very important to us. Please read each of the following questions and provide us with your 
response. To maintain the confidentiality of your response, please place the survey in the 

accompanying envelope, seal it and return it to . 

*************************************************************************** 

Gender: Male Female 

Ethnicity: 

Administrative Position: ' 

Number of Years as an administrator : 



Number of Years the Science Research Program has been in the district: 



Support for the Science Research Program 

The following are areas in which support may have been provided for the implementation of the 
Science Research Program. For each, please circle the number indicating the level of support 
provided or received. 



Outstanding Sufficient Insufficient No Support 



1. Adjustment of teacher load. 1 

2. Establishment of communication 

networks. 1 

3. Teacher computer for DIALOG, 

email, and internet. 1 

4. Long distance phone line. 1 

5. Course list in course catalog. 1 

6. Support from other teachers. 1 

7. Support from school counselors. 1 

8. Support from administrators. 1 

9. Support from mentors in the 

community. 1 

10. Support from parents. 1 

1 1 . Support from school board. 1 

12. Support from State Education. 1 



2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 



3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 



4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 



Selection/Retention Process 



For each facet of the selection/retention process, circle the number indicating your impression of 
how well the process is functioning. 



Working 

Well 

1 . Recruitment of teachers into 

the program 1 

2. Identification of prospective 

students for the program 1 

3. Spring recruitment meeting 

Freshman year 1 

4. Fall meeting with selected 

students 1 

5. Process of retaining 

students in program. 1 



Is 


Needs Extra 


Not Area 


Adequate 


Work 


Emphasis 


2 


3 


4 


2 


3 


4 


2 


3 


4 


2 


3 


4 


2 


3 


4 



For what type of student is this program most appropriate? 



How was the teacher who works with this program selected? 



Communications 

Please rate your perception of the communication amongst the various program participants. 



Outstanding Sufficient 



Insufficient Not Happening 



1. Student & Teacher 1 

2. Student & Mentor 1 

3. Teacher & Mentor 1 

4. Teacher & Administrator 1 

5. Teacher & School Counselor 1 

6. School & Parent 1 



2 3 4 

2 3 4 

2 3 4 

2 3 4 

2 3 4 

2 3 4 



About the Science Research Program 

The following statements are about the Science Research Program. Please indicate your level of 
agreement with each statement by circling the number that best represents your opinion. 
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l=strongly agree; 2=agree; 3=slightly agree; 4=slightly disagree; 5=disagree; 6=strongIy 

disagree 



1. The program has improved students’ 
knowledge about conducting 
science research 

2. The program has increased students’ 
knowledge about science. 

3. The program is interesting to teach. 

4. The program has helped students’ 
to improve: 

a. creative thinking ability. 

b. problem solving skills 

c. communication skills. 

d. time management skills. 

5. Due to the program, student motivation 
for learning has increased for: 

a. science 

b. other school subjects 

6. Students’ self-esteem has increased 
due to the program. 

7. The class size is appropriate. 

8. The program challenges students. 

9. The program has changed the 
concept of science education 

10. 1 support continuation of the 
program 



Agreement Scale 



SA 

1 2 3 



1 2 3 

1 2 3 



1 2 3 
1 2 3 
1 2 3 
1 2 3 



1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 

1 2 3 



SD 

4 5 6 



4 5 6 

4 5 6 



4 5 6 
4 5 6 
4 5 6 
4 5 6 



4 5 6 

4 5 6 

4 5 6 

4 5 6 

4 5 6 

4 5 6 

4 5 6 



Please check any of the following that you believe students have learned or improved as a direct 
result of participation in the Science Research Program. 

reading for meaning effective use of technology 

problem solving interpretation of data 

reasoning skills real life application of science 

writing a report/abstract outlining or notetaking 

organization and planning skills summarizing or paraphrasing 

thinking creatively collecting data 

communicating effectively orally reading graphs and charts 

communicating effectively in writing time management 

identifying the difference between a fact and an opinion 

making an inference - using an observation to. make a prediction 

anticipating possible problems and finding the solution 



Of the above check-list, which do you think are most unique to this program? 



How do you see this program impacting your overall curriculum and resources? 

Thank you. 



Appendix B 

Student, Teacher and Administrator Interview Protocols 
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Science Research Program 

Student Interview Protocol 



School District: 

Years with Science Research Program: 

Grade: Gender: Ethnicity: 

Interviewer: 

Date of interview: 



Interview overview: 

• probe for detail; get specific examples 

• search for meanings, interpretations, perceptions on part of interviewee 
Potential probes: 

• barriers 

• supports 

• contrast ideal and current reality 

• factors that prevent ideal; are these doable, in your control 
Questions. 

• If you were to tell another student about the SRP, how would you describe it? 

• What is the Program’s purpose, goals, and who should take it? 

• How did you become involved in the Program? 

• What was interesting about it? 

• How did you find out about it? 

• How were you picked to be a part of the Program? What did you have to do? 

• What benefits or opportunities does this class provide that other science classes don’t? 

• How is this class different from your other classes, in general? 

• Have your future plans changed as a result of your involvement with the program, i.e., has this 
course influenced your intended area of specialization in college, or work interests? 

• Describe your experience in the course and conducting research. 

• What joys did you experience? Frustrations? 

• Of your sophomore year activities, which were easy for you and which were hard, i.e., 
mastering DIALOG, literature review, presenting articles, hypothesis generation, starting 
mentor relationship? 

• Of your junior year activities, which were easy and which were hard, i.e., communicating with 
mentor, continued literature review, meeting with teacher, presentation of initial findings, 
introduction to statistical hardware? 

• Describe the degree of assistance you have received from your teacher and also your mentor. 
What kinds of help did you get? In what areas would you like additional help? 
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Science Research Program 

Teacher Interview Protocol 



School District: 

Years with Science Research Program: Years Teaching: 

Grade: Gender: Ethnicity: 

Interviewer: 

Date of interview: 



Interview overview: 

• probe for detail; get specific examples 

• search for meanings, interpretations, perceptions on part of interviewee 
Potential probes: 

• barriers 

• supports 

• contrast ideal and current reality 

• factors that prevent ideal; are these doable, in your control 
Questions: 

• If you were to tell another teacher about the SRP, how would you describe it? 

• What is the program’s purpose, goals and whom does it serve? 

• How did you get students recruited and selected into the program? 

• What do you see as the key components of the course curriculum? 

• How is student progress assessed? Describe a biweekly meeting (progress checklist, portfolio 
review) 

• What are the benefits of the program for the student? 

• Now that our are one/two years into the program, how useful was the summer training? 

What did you learn that you were able to utilize in the classroom? 

In what areas would you have liked more information? 

What single item was the most helpful? 

What would you change about the workshop, either format or content? 

• How useful have the teacher support group meetings been? 

What have you learned that you have been able to utilize? 

In what areas do you continue to need more information? 

• What single item was the most helpful? 

• How would you change the meetings, either format or content? 

• What is the degree of support for the program you have experienced from your district? 
Administration? Home department? Peers or other teachers? Parents and the community? 

• How well is the program integrated into your building and school curriculum? 

• On a day-to-day basis, what activities hinder or facilitate the implementation of the program? 
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Science Research Program 

Administrator Interview Protocol 



School District: Position: 

Years with district: 

Gender: Ethnicity: 

Interviewer: 

Date of interview: 



Interview overview: 

• probe for detail; get specific examples 

• search for meanings, interpretations, perceptions on part of interviewee 
Potential probes: 

• barriers 

• supports 

• contrast ideal and current reality 

• factors that prevent ideal; are these doable, in your control 
Questions: 

• If you were to tell another administrator about the SRP, how would you describe it? 

• What is the program’s purpose, goals, and who does it serve? 

• How are students recruited and selected for the program? 

• What kinds of students are participating? 

• Who is best served by the program? 

• How is this program meeting the needs of the student body as a whole? 

• What is the impact of student attrition from the program? 

• What changes have been made to implement the program? What is yet to be provided? 

• What are the advantages and drawbacks of the program? 

• What is the long-term viability of the program? 

• How has the program impacted your overall science curriculum? 

• How does the program fit in with NYS educational standards? 

• How is the program different from others classes that are offered? 
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Appendix C 

Symposia Observation Form 



Science Research Program 
Symposium Observation 



School District: 

Date of Symposium: 

Name of Observer: 

Number of students in the program: 

Number of students participating in the symposium, j 

People attending the symposium: 

parents classmates mentors school board 

siblings teachers administrators other: 

Format of the student presentations: 



Oral Presentations 
Number of oral presentations: 



Approximate length of oral presentations: 


minutes 






Comments: 


Did the students use slides and/or graphic analyses in their presentations? 


Yes 


No 




Did the students clearly convey their: 


purposes? 


Yes 


No 






materials? 


Yes 


No 






methods? 


Yes 


No 






results? 


Yes 


No 






discussions? 


Yes 


No 




. % 


conclusions? 


Yes 


No 




Poster Presentations 
Number of poster presentations: 

Was the information presented word-processed? 




Yes 


No 


Comments: 


Did the students use slides and/or graphic analyses in their presentations? 


Yes 


No 




Did the students clearly convey their: 


purposes? 


Yes 


No 






materials? 


Yes 


No 






methods? 


Yes 


No 






results? 


Yes 


No 






discussions? 


Yes 


No 






conclusions? 


Yes 


No 




Goals 










Strongly Agree = 1 ; Agree = 2; Disagree = 3; Strongly Disagree = 4 




Based on the presentations, did the students: 










1) use the scientific method? 


1 2 


3 


4 




2) draw conclusions based on data? 


1 2 


3 


4 




3) propose future research? 


1 2 


3 


4 




4) conduct literature searches? 

5) conduct authentic experimentation which 


1 2 


3 


4 




addresses hypotheses, is reproducible and 
has appropriate controls? 


1 2 


3 


4 




6) collect, organize, analyze and interpret data? 


1 2 


3 


4 




7) work with a professional mentor? 


1 2 


3 


4 





Other Comments/Observations (continue on back): 



Appendix D 

Cover Letter for Surveys 
Confirmation Letter for Site Visits 



Evaluation Consortium at Albany 
School of Education 



Education BIO 
Albany, New York 12222 




University at Albany 

STATE UNIVERSITY OF NEW YORK 



June 2, 1997 



Dear Teacher, 



We have been contracted to gather information regarding the Science Research Program 
in various school districts in New York state. As a district that has participated in the National 
Science Foundation training, your input is integral to this endeavor. We are interested in 
obtaining the impressions of the stakeholders in this program. To that end, we have enclosed 
surveys to be completed by all students in the program, as well as the teacher and administrators 
involved. We have provided copies of the surveys. Please make additional copies as necessary. 

In addition, we would appreciate your assistance in gathering the following information 
that is important to the evaluation process. 



1 . course outlines 

2. recruitment brochures/handouts 

3. support group meeting minutes (if available) 

4. demographic information of the students and teachers in 
your district and the program (please use attached form) 



Thank you for your efforts in gathering the above information. Please return the surveys 
and supporting materials in the enclosed envelope by June 16, 1997. Should you have any 
questions, please feel free to call us at (518) 442-5027. 



Sincerely, 

Lucy Cardella 
Laura Zebrowski 
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Evaluation Consortium at Albany 
School of Education 




Education BIO 
Albany, New York 12222 

518/442-5027 



University at Albany 

STATE UNIVERSITY OF NEW YORK 



June 2, 1997 



Dear Teacher, 



We appreciate the opportunity to observe the Science Research Program at your school 
and to meet with the participants. Per our telephone conversation, our visit is scheduled for June 
1 1, 1997. We anticipate arriving at 1 1 :00 a.m.. 

During our visit, we would like to interview the following people to gather their 
perceptions of the program: 

1. you, the Science Research Program Teacher 

2. the Building Administrator (or a central administrator or science department 
chairperson, if the principal is not available) 

3. at least one group of 3 to 4 students involved in the program 

We anticipate that these interviews will last 45 minutes to one hour 

In addition, we would like an opportunity to view the students portfolios as well as the 
class in session. Your support in gathering the following information would greatly assist us in 
the evaluation process: 

1 . course outlines 

2. recruitment brochures/handouts 

3 . support group meeting minutes (if available) 

4. demographic information of the students and teachers in 
your district and the program (see attached form) 

5. the enclosed surveys to be completed by the students, 
teacher, and administrator 

We appreciate your efforts in arranging the details of our visit and gathering this 
information. Should you have any questions, please feel free to call us at (518) 442-5027. We 
look forward to our visit. 





Sincerely, 

Lucy Cardella 
Laura Zebrowski 



Appendix E 

Assessment Rubrics 



Check Sheet for Research Course Implementation Success 



School Teacher 

Date course began Date of Assessment 

Number of students in class: Year 1 Year 2 Year 3 







Yes 


No 


1 


Was there a spring meeting with prospective students and their 
parents (or if course implementation was too late for a spring 
meeting, was there a fall meeting)? 






2 


By midyear, did the teacher begin a list of prospective students to 
recruit for the next year? 






3 


Was the teacher available to students during the summer? 






4 


Did new students have sufficient articles to discuss, from 
common literature, at the start of school (ten articles from 
assorted sources)? 






5 


Were there five completed biweekly assessment sheets for each 
student, each quarter? 






6 


Did the student/teacher conferences begin by the third week of 
school? 






7 


Did each student complete three or more successful DIALOG 
searches, including author searches, by mid winter? 






8 


Have all new studnets contacted a prospective mentor by April of 
their first year in the course? 






9 


Did every student know his/her role in the upcoming symposium 
by mid-April? 






10 


Were all sections of each student’s portfolio kept up to date? 






11 


Did the teacher provide opportunities for students to make out-of- 
class presentations on their topics? 






12 


Did the teacher require students to attend available science 
symposia and presentations? 






13 


Did each student produce a quality poster by the end of year one 
(includes graphics, word processing, layout)? 






14 


Did students enroll for college credit in their second and third 
years? 






15 


Did second and third year students enter competitions? 






16 


Did every student participate in the end of year symposium? 







School 

Date 



Science Research Program 

Portfolio Review 



# of Portfolios Reviewed . 
Demographic Information 



Contents none minimal 



• Lab Notebook 1 2 

• Goal Sheets 1 2 

• Email/Intemet/DIALOG Log 1 2 

• Journal Articles 1 2 

• Telephone Log 1 2 

• Meetings Log 1 2 

• Assessment 1 2 

• Grants 1 2 

• Presentations 1 2 

• Honors/Awards 1 2 

• Abstracts 1 2 

• Final Paper 1 2 

• Rules for Competitions 1 2 

• Log of Time Commitments 1 2 

• Essay on Excellence 1 2 

• Essay on Research Direct. 1 2 



normative 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 



superior NA 

4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 
4 



Impressions 

General 



Portfolio #1 



Portfolio #2 
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Documentation Checklist 
Demographic Profiles 
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School District 



Documentation Required from Each Site Science Research Programs 



Course Outlines 

Recruitment Brochures/Handouts 
Portfolio Checklists 

Support Group Meeting Minutes (if available) 

Demographics 
District wide 

Of students in the program & teachers in the program 

Completed Surveys 

Students 

Teachers 

Administrators 

Completed Interviews 

Students 

Teachers 

Administrators 




5 



2 



School district: 
County: 



DEMOGRAPHIC PROFILES 

DISTRICT - Student Enrollment 

Total # 

% White 

% Black 

% Hispanic 

% Other 

% Male 

% Female 

Youth at Risk 

Census Poverty Index 

% Free/Reduced Lunch 

% Limited English 

Dropout Rate 

Suspension Rate ■ 

High School Graduates 

1995-1996 % Regents Diploma 

1995-1996 % to College 

Classroom Teachers 

Pupil to Teacher Ratio 

% Minority 

Median Years of Experience 

% Male 

% Female 




Date 
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SCIENCE RESEARCH PROGRAM - Students in Program 

Total # __ 

% White 

%Black 

%Hispanic 

%Other 

%Male 

%Female 

%with disabilities 

%Limited English 

%College Bound 

%Regents Diploma Track 

%Concurrently Taking Another Science Course 

%Free/R*educed Lunch 



Dropout Rate 

Suspension Rate 

Program Teachers 

Pupil to Teacher Ratio 

Ethnic Background: 

White 

Black 

Hispanic 

Other 

# Years Teaching Experience 
Gender 
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